Actually both x_i and x_j are support vectors because our demand is to reduce the Lagrangian. You can check it more precisely on Professor Berwick’s notes

http://web.mit.edu/6.034/wwwbob/svm-notes-long-08.pdf