Return the lowest value
Check out Analytical functions. Here's how you could solve your problem using the RANK() analytical function in Oracle (for simplicity I select from a table called "courses" and not your subquery; the table has the same structure as the one in your picture):

select * from (
    select coursedept, 
           coursenum, 
           numstudent, 
           rank() over(partition by coursedept order by numstudent) num_s_rank 
     from courses
 ) where num_s_rank = 1;


Note that in case of ties multiple rows get the same rank, which means you could get more than one course per department.