Elder abuse is a serious public health concern that increases the risks of negative health outcomes globally. It is well known that abuse in older adults is associated with depression; however, longitudinal studies investigating the causal relationship between these events are scarce. Because the cause precedes the result over time, the temporal relationships between abuse and depression should be verified from each direction. This longitudinal study, therefore, investigated and clarified whether depression causes or results from elder abuse among older Japanese adults. Two longitudinal analyses were conducted using data derived from the Japan Gerontological Evaluation Study. The data were collected in 2010 and 2013 through a mail survey of 1,737 people (983 females, 754 males) across Japan. Of those who did not experience abuse in 2010, 38 (5.0%) males and 53 (5.4%) females newly experienced abuse in 2013. Among respondents who did not have depressive symptoms in 2010, 60 (8.0%) males and 61 (6.2%) females newly reported depressive symptoms in 2013. After adjusting for demographic factors in Analysis 1, people who experienced abuse were 2.28-fold (95% confidence interval [CI] = 1.68-3.09) more likely to have depressive symptoms three years later than those who were not abused. In Analysis 2, respondents who had mild or severe depression in 2010 were 2.23-fold (95% CI = 1.61-3.10) more likely to have experienced abuse after three years than those who did not have depression. After adjusting for several demographic factors, the results showed that abuse can lead to depression and that depression can be a cause of abuse. Therefore, preventing abuse should be considered from both directions.