C++ wrapper class for array of OpenGL buffer objects -- just the constructors
Some specific questions I have about this code/approach are, if applicable,

OK.

Error handling design decision: should the constructors throw, or should I leave error checking, if desired, to the caller?

Either the object is correctly initialized and ready to use or it should throw. Two stage initialization (construct then check) is a bad idea as it leaves it open to the user to do the correct thing. You should make sure your object can not be abused not relie on the user to not abuse you.

Design decision: is it a good idea to use templates here?

If you are going to use std::array you don't really have a choice.

Will this lead to problematic code bloat if I'm using a lot of different-sized buffer_objects, or is it efficient? Can I evaluate this through profiling?

Potentially this will lead to methods be compiled for different types. But this is compiler dependent and some modern compiler can optimize this out. But what are you comparing it too?

Are the memcpys worth it to put the rvalues in a valid non-owning state, or can I treat the rvalues as non-owning leftovers?

I think you need to null out the src values. Otherwise the destructor is going to release the names. Alternatively you can store more state about if the object is valid and make the call to glDeleteBuffers() conditional on the object being in a valid state.
But I also think there is a bug here. You have just copied over the values. But you did not release the values on the dst side before the copy so you have lost the names stored there.
Remember that a std::array does not have its own move (as the data is local to the object not dynamically allocated). It moves the underlying objects between containers.

Am I properly using the base class's move constructor?

Yes it seems ligit.
Suggestions.
I would make the std::array a member rather inherit from it.
I would do:
#ifndef THORSANVIL_GL_WRAPPER_H
#define THORSANVIL_GL_WRAPPER_H

#include &lt;GL/gl.h&gt;
#include &lt;array&gt;
#include &lt;algorithm&gt;
#include &lt;cstring&gt;

namespace ThorsAnvil::GL {

template&lt;size_t N&gt;
class BufferObjects
{
    std::array&lt;GLuint, N&gt;  buffer;
    bool                   valid;
    public:
        BufferObjects() noexcept;
       ~BufferObjects();

        // Note: These are non copyable objects.
        //       Deleting the copy operations.
        BufferObjects(BufferObjects const&amp;)           = delete;
        BufferObjects operator=(BufferObjects const&amp;) = delete;

        // Note: The move is as expensive as a copy operation.
        //       But we are doing a logical move as you 
        //       can not have two objects with the same resource.
        BufferObjects(BufferObjects&amp;&amp; from)            noexcept;
        BufferObjects&amp; operator=(BufferObjects&amp;&amp; from) noexcept;

        // Reset an invalid object.
        // Note: If object is valid no action.
        void reset();

};

template&lt;size_t N&gt;
BufferObjects&lt;N&gt;::BufferObjects() noexcept
    : valid(false)
{
    reset();
}

template&lt;size_t N&gt;
BufferObjects&lt;N&gt;::~BufferObjects()
{
    if (valid) {
        glDeleteBuffers(N, buffer.data());
    }
}

template&lt;size_t N&gt;
BufferObjects&lt;N&gt;::BufferObjects(BufferObjects&lt;N&gt;&amp;&amp; from) noexcept
{
    // Move the resources from the other object.
    std::move(std::begin(from.buffer), std::end(from.buffer), std::begin(buffer));

    // Maintain the correct valid states
    // The rhs is no longer in a valid state and has no resources.
    valid = from.valid;
    from.valid = false;
}

template&lt;size_t N&gt;
BufferObjects&lt;N&gt;&amp; BufferObjects&lt;N&gt;::operator=(BufferObjects&lt;N&gt;&amp;&amp; from)
    noexcept
{
    // The standard copy and swap not efficient.
    // So we should do a test for self assignment
    if (this != &amp;from)
    {
        // Destroy then re-create this object.
        // Call destructor and then placement new to use
        // the move copy constructor code to set this value.
        ~BufferObjects();
        new (this) BufferObjects(std::move(from));
    }
    return *this;
}

template&lt;size_t N&gt;
void BufferObjects::reset()
{
    if (!valid) {
        glGenBuffers(N, buffer.data());
        valid = true;
    }
}