We report on an interesting mechanism of inducing chiroptical response at plasmonic silver nanoparticles (NPs) through the formation of plasmonic hot spots in small metal-NP-chiral-surfactant assemblies. Circular dichroism (CD) was measured at the surface plasmon resonance of cholate-coated silver nanostructures (AgCT) in the visible region of the spectrum. Low temperature cryogenic transmission electron microscopy (cryo-TEM) micrographs of the AgCT nanostructures in solution reveal small assemblies of silver NPs. Upon pH increase these assemblies are separated into individual NPs, and the induced plasmonic CD vanishes. This process was monitored via spectroscopy (CD and absorption), cryo-TEM, small-angle X-ray scattering (SAXS), and dynamic light scattering (DLS) measurements. The synthesis of well-separated AgCT NPs, which was performed with a large excess of sodium cholate (NaCT), also did not show any chiroptical effects. We interpret and model the formation of strong CD signals in the visible range in terms of the molecule-plasmon interaction in plasmonic hot spots formed in nanoparticle aggregates. Importantly, this study of the chiral induction, transfer to the visible range, and local field enhancement offers very attractive possibilities for sensing and detection of chirality of small amounts of molecules using visible light.