Custom component organization system
+1 to @Gilbert Le Blanc's comment. Creating 

I'd consider extracting out methods like createOpenMenuItem(), createSaveMenuItem(), createPrintMenuItem() for better readability and higher abstraction:

private JMenuItem createOpenMenuItem() {
    JMenuItem openMenuItem = new JMenuItem("Open");
    openMenuItem.addActionListener(new ActionListener() {
        @Override
        public void actionPerformed(ActionEvent e) {

        }
    });
    return openMenuItem;
}


Then the setUpFileMenu would be a lot cleaner:

private void setUpFileMenu() {
    fileMenu = new JMenu("File");

    // File menu items
    fileMenu.add(createNewMenuItem());
    fileMenu.add(createOpenMenuItem());
    fileMenu.add(createSaveMenuItem());
    fileMenu.add(createSaveAsMenuItem());
    fileMenu.add(createPrintMenuItem());

    add(fileMenu);
}


A maintainer would be helpful, it's easier to get an overview what the method does (without the details) and you can still check them if you are interested in. For example, finding code of save menu is much faster than before.


  Now, each JMenuItem obviously needs an ActionListener, 
  so I just gave each one a different ActionListener. 
  I'm skeptical about this because I know it can be kind of disorganized looking. 


I've found this practice easier to work with than having one ActionListener for multiple components (with a switch-case inside).


  Basically, I just want somebody to let me know
  if this practice is all right, or if there is a better 
  alternative that doesn't involve me making 20 new classes.


Having a lot of classes doesn't hurt if they have clear responsibilties.