Unfortunately there is no direct event on the ScrollViewer to detect change on the scrollbar. You need to get at the children directly. Here's how to detect change on a vertical scrollbar in the ScrollViewer

ScrollBar verticalScrollBar = ((FrameworkElement)VisualTreeHelper.GetChild(viewer, 0)).FindName("VerticalScrollBar") as ScrollBar;
verticalScrollBar.ValueChanged += (s, ev) => button.Visibility = System.Windows.Visibility.Collapsed;

Don't add this code in the constructor, put it in the Loaded event handler of the child contained in the ScrollViewer.