2

Using Google Drive API, I am trying to upload files to Google Drive.

Please see code below. The code runs successfully but instead of uploading all files to one folder, it is creating separate folder to each file it uploads.

I believe I am going wrong with this line in the code: q="name='" + folder + "' and trashed=false and mimeType = 'application/vnd.google-apps.folder' and '{0}' in parents".format(parent_folder_id) but couldnt figure out. Any help is highly appreciated.

def uploadFile(service,pathname, mimetype, folder_id,replace):
    # Check if the file exists on google drive
    filename=os.path.split(pathname)[1]
    page_token = None
    q="name='" + filename + "' and trashed=false  and '{0}' in parents".format(folder_id)
    try:
        response = service.files().list(q= q, spaces='drive',
                                          fields="files(name, id)",
                                          pageToken=page_token).execute()
        if replace==True:
            file_id=response.get('files')[1]['id']
            file = service.files().get(fileId=file_id).execute()
             # File's new content.
            media_body = MediaFileUpload(pathname, resumable=True)
             # Send the request to the API.
            updated_file = service.files().update(
                fileId=file_id,
                body=file,
                newRevision=False,
                media_body=media_body).execute()
    except:
        file_metadata = {'name': filename, "parents": [folder_id]}
        media = MediaFileUpload(pathname,
                                mimetype=mimetype)
        file = service.files().create(body=file_metadata,

                                      media_body=media,
                                      fields='id').execute()
#################################
def UploadFiles(inifile, filespec, parent_folder):
    page_token = None
    # Retrieve the parent folder id from the parent folder name
    q="name='" + parent_folder + "' and trashed=false and mimeType = 'application/vnd.google-apps.folder'"
    try:
        response = service.files().list(q=q, spaces='drive',fields="files(name,id)",pageToken=page_token).execute()
    except:
        print("Parent folder",parent_folder,"was not found")
        return
    parent_folder_id = response.get('files')[0]['id']
    ##For loop to go through each file in filelist to upload
    for pathname in glob.glob(filespec):
    #for files in file_list:
        folder=pathname.split('_')[1]
        # Check if the folder exists
        q="name='" + folder + "' and trashed=false and mimeType = 'application/vnd.google-apps.folder' and '{0}' in parents".format(parent_folder_id)
        try:
            response = service.files().list(q=q, spaces='drive',fields="files(name,id,parent_id)").execute()
            folder_id = response.get('files')[0]['id']
        except:
            #if folder doesnt exist, create and upload
            file_metadata = {'name': folder,'mimeType':'application/vnd.google-apps.folder','parents':[parent_folder_id]}
            #createFolder(folder)
            new_folder = service.files().create(body=file_metadata, fields='id').execute()
            folder_id = new_folder.get('id')
        # Now upload the file!
        if pathname.endswith("pdf"):
            mimetype = 'application/pdf'
        else:
            mimetype = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
        #print("uploading file ",files," path",filepath," filename",filename, " folderid", folderid)
        uploadFile(service,pathname,mimetype,folder_id,True)

2 Answers 2

1

I am not a python dev, so you may have to work on the code a bit.

There is an issue with your search

#Check if the folder exists
q="name='" + folder + "' and trashed=false and mimeType = 'application/vnd.google-apps.folder' and '{0}' in parents".format(parent_folder_id)

The correct format is parents in 'ParentFolderId' which should give you.

q="name='" + folder + "' and trashed=false and mimeType = 'application/vnd.google-apps.folder' and parents in '{0}' in ".format(parent_folder_id)

Also remember that service.files().create will create a new file every time its called if you want to update an existing file user files().update

Sign up to request clarification or add additional context in comments.

2 Comments

I tried with parents in '{0}' in ".format(parent_folder_id) but same issue that it creates multiple folders.
Im not the best at Python but i would check the response comming back from response = service.files().list(q=q, spaces='drive',fields="files(name,id,parent_id)").execute() Is it actually returning something and your reading it wrong?
0

I've encountered a similar problem on a project, and I've adapted it to fit your needs for Google Drive uploading with dummy data below:

from googleapiclient import discovery
from googleapiclient.http import MediaFileUpload, MediaIoBaseDownload
from httplib2 import Http
from oauth2client import file, client, tools
from pathlib import Path
import datetime

def gDriveFolder(service, name: str):
        Drive_service = service

        #Check if GDrive folder exists
        folder_metadata = {
            'name': name,
            'mimeType': 'application/vnd.google-apps.folder'
        }
        query = "name contains '{}' and mimeType='{}' and trashed=false".format(folder_metadata["name"],folder_metadata["mimeType"])
        response = Drive_service.files().list(q=query, pageSize=100, fields="nextPageToken, files(id, name, mimeType, size, parents, modifiedTime)").execute().get('files', [])
        if response.__len__() == 1 and response[0].get("name") == folder_metadata["name"]:
            folder_id = response[0].get("id")
        else:
            #creates GDrive folder
            gfolder = Drive_service.files().create(body=folder_metadata,
                                            fields='id').execute()
            folder_id = gfolder.get('id')
        
        return folder_id


def gDriveFolderFilesQuery(service, folder_name_for_q : str):
    Drive_service = service
    folder_id = gDriveFolder(Drive_service, folder_name_for_q)
    folder_query = "'%s' in parents" % folder_id 
    files_response = Drive_service.files().list(q=folder_query, pageSize=100, fields="nextPageToken, files(id, name, mimeType, size, parents, modifiedTime)").execute().get('files', [])
    file_names = dict()
    for log_file in files_response:
        file_names[log_file.get("name")] = log_file.get("id")
    return file_names

def uploadFile(service, pathname, mimetype_v, folder_id, folder_name_for_q, replace):
        Drive_service = service

        file_names = gDriveFolderFilesQuery(Drive_service, folder_name_for_q)

        dummy_file_path = Path(pathname)
        file_name = pathname.name
        
        if not mimetype_v:
            mimetype_v = 'text/plain'
        
        if replace:
            file_metadata = {
                'name': file_name,
                'mimeType': 'application/vnd.google-apps.script',
                'parents':[folder_id]
            }
            media = MediaFileUpload(dummy_file_path,
                                    mimetype=mimetype_v,
                                    resumable=True)
            file = Drive_service.files().create(body=file_metadata,
                                                media_body=media,
                                                fields='id').execute()

        #checks if the text.txt exists
        if file_name in list(file_names.keys()):
            file_id = file_names[file_name]
            media = MediaFileUpload(dummy_file_path,
                                    mimetype=mimetype_v,
                                    resumable=True)
            file = Drive_service.files().update(fileId=file_id,body=None,
                                                media_body=media).execute()

        else:
            file_metadata = {
                'name': file_name,
                'mimeType': 'application/vnd.google-apps.script',
                'parents':[folder_id]
            }
            media = MediaFileUpload(dummy_file_path,
                                    mimetype=mimetype_v,
                                    resumable=True)
            file = Drive_service.files().create(body=file_metadata,
                                                media_body=media,
                                                fields='id').execute()

def uploadFiles(service, DirectoryFolder, mimetype_v, folder_id, folder_name_for_q, replace):

    for a_file in DirectoryFolder.glob("*"):
        if a_file.is_dir():
            continue
        uploadFile(service, a_file, mimetype_v, folder_id, folder_name_for_q, replace)



if __name__ == "__main__":
    
    SCOPES = ['https://www.googleapis.com/auth/drive', 
                        'https://www.googleapis.com/auth/drive.file', 
                        'https://www.googleapis.com/auth/drive.appdata',
                        'https://www.googleapis.com/auth/drive.metadata', 
                        'https://www.googleapis.com/auth/drive.activity']


    store = file.Storage('storage.json')
    creds = store.get()
    if not creds or creds.invalid:
        flow = client.flow_from_clientsecrets('client.json', SCOPES)
        creds = tools.run_flow(flow, store)
    DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http())) #be prepared to be authorized

    #make a dummy folder
    dummy_file_directory = Path("dummy/")
    dummy_file_directory.mkdir(parents=True, exist_ok=True)

    #create a set list of dummy files
    how_many_files = 5
    for i in range(how_many_files+1):
        dummy_file = dummy_file_directory / ("test_num_%s.txt" % i)
        data = "current date and time for iteration %s is %s" % (i, datetime.datetime.now())
        with open(str(dummy_file), "w", encoding='utf-8') as f:
            f.write(data) 

    folder_name_for_q = "folder_for_priya"
    folder_id = gDriveFolder(DRIVE, folder_name_for_q)

    mimetype_v = 'text/plain'
    uploadFiles(DRIVE, dummy_file_directory, mimetype_v, folder_id, folder_name_for_q, replace=False)

I'll do my best to give a quick rundown of what's going on. Running the main program, credentials and scopes are declared. (This method of authorization was adapted from the following guide)

Next a dummy folder is created and later populated with dummy files (in this instance 5 text files) with their respective iterations and timestamps with the datetime module.

Afterwards a new variable is defined to give a name of a folder to be on google drive.

The method gDriveFolder, checks if such a GoogleDrive directory already exists. Regardless if such a folder exists, the folder id of the folder is returned.

Next the method uploadFiles is called, where it takes the pathlib.Path object passed within it, and iterates over the files that it detects that are not possibly a folder. For each file within the folder dummy_file_directory, the method uploadFile is executed.

Finally, uploadFile takes the file with it's respective mimetype, and checks if such a file exists by first calling the gDriveFolderFilesQuery method and validates if the uploading file is already in the GoogleDrive directory.

Hopefully, the walkthrough was simple enough to materialize the implementations you intended.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.