dopetalk does not endorse any advertised product nor does it accept any liability for it's use or misuse

This website has run out of funding so feel free to contribute if you can afford it (see footer)

Author Topic: Subdominant Dense Clusters Allow for Simple Learning and ...  (Read 1645 times)

Offline Chip (OP)

  • Server Admin
  • Hero Member
  • *****
  • Administrator
  • *****
  • Join Date: Dec 2014
  • Location: Australia
  • Posts: 6514
  • Reputation Power: 0
  • Chip has hidden their reputation power
  • Gender: Male
  • Last Login:December 02, 2021, 05:51:37 AM
  • Deeply Confused Learner
  • Profession: IT Engineer
source: https://journals.aps.org/prl/pdf/10.1103/PhysRevLett.115.128101

Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses [March 2015]

We show that discrete synaptic weights can be efficiently used for learning in large scale neural systems,
and lead to unanticipated computational performance. We focus on the representative case of learning random
patterns with binary synapses in single layer networks. The standard statistical analysis shows that this
problem is exponentially dominated by isolated solutions that are extremely hard to find algorithmically.
Here, we introduce a novel method that allows us to find analytical evidence for the existence of subdominant
and extremely dense regions of solutions. Numerical experiments confirm these findings. We also show that
the dense regions are surprisingly accessible by simple learning protocols, and that these synaptic
configurations are robust to perturbations and generalize better than typical solutions. These outcomes
extend to synapses with multiple states and to deeper neural architectures. The large deviation measure also
suggests how to design novel algorithmic schemes for optimization based on local entropy maximization.

the article continues on via the source link ...
« Last Edit: March 12, 2020, 07:51:37 PM by Chip »
friendly
0
funny
0
informative
0
agree
0
disagree
0
like
0
dislike
0
No reactions
No reactions
No reactions
No reactions
No reactions
No reactions
No reactions
I do not condone or support any illegal activities. All information is for theoretical discussion and wonder.
All activities discussed are considered fictional and hypothetical. Information of all discussion has been derived from online research and in the spirit of personal Freedom.

Tags:
 

Related Topics

  Subject / Started by Replies Last post
0 Replies
261 Views
Last post April 27, 2015, 02:43:51 AM
by smfadmin
4 Replies
3580 Views
Last post February 17, 2018, 06:46:56 AM
by dillydudeEL14
0 Replies
2017 Views
Last post June 02, 2018, 10:10:59 AM
by Chip
0 Replies
2056 Views
Last post June 27, 2018, 05:27:39 AM
by Chip
0 Replies
2210 Views
Last post June 29, 2018, 02:15:06 PM
by Chip
0 Replies
1748 Views
Last post September 24, 2018, 01:43:39 PM
by Chip
0 Replies
1399 Views
Last post June 17, 2019, 04:51:50 PM
by Chip
0 Replies
1145 Views
Last post July 06, 2019, 09:31:56 AM
by Chip
0 Replies
1247 Views
Last post July 07, 2019, 07:12:28 AM
by Chip
0 Replies
1578 Views
Last post September 06, 2019, 07:30:28 AM
by Chip


dopetalk does not endorse any advertised product nor does it accept any liability for it's use or misuse





TERMS AND CONDITIONS

In no event will d&u or any person involved in creating, producing, or distributing site information be liable for any direct, indirect, incidental, punitive, special or consequential damages arising out of the use of or inability to use d&u. You agree to indemnify and hold harmless d&u, its domain founders, sponsors, maintainers, server administrators, volunteers and contributors from and against all liability, claims, damages, costs and expenses, including legal fees, that arise directly or indirectly from the use of any part of the d&u site.


TO USE THIS WEBSITE YOU MUST AGREE TO THE TERMS AND CONDITIONS ABOVE


Founded December 2014
SimplePortal 2.3.6 © 2008-2014, SimplePortal