Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(chat-e2e): updated tests with stateful api usage #683

Merged
merged 37 commits into from
Feb 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
92675f7
feat/update-e2e-with-stateful-api: updated tests with stateful api usage
irinakartun Feb 13, 2024
f8e7f97
feat/update-e2e-with-stateful-api: fixed file api endpoint
irinakartun Feb 13, 2024
d35180c
feat/update-e2e-with-stateful-api: fixed flaky tests
irinakartun Feb 13, 2024
89042d8
Merge branch 'development' of github.com:epam/ai-dial-chat into feat/…
irinakartun Feb 13, 2024
19b6092
feat/update-e2e-with-stateful-api: fixed flaky test
irinakartun Feb 13, 2024
c40ba5b
feat/update-e2e-with-stateful-api: enabled part of chat export tests
irinakartun Feb 13, 2024
60d2793
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 13, 2024
a5db64d
feat/update-e2e-with-stateful-api: fixed return
irinakartun Feb 13, 2024
fe099b2
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 13, 2024
acd6ad9
feat/update-e2e-with-stateful-api: adjusted items path to latest changes
irinakartun Feb 14, 2024
1ec5203
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
f2849fd
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
74465cb
feat/update-e2e-with-stateful-api: fixed prompt update method
irinakartun Feb 14, 2024
3d17060
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
fca94b1
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
8412955
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
fe7e8b8
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
c98b395
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
28912f4
feat/update-e2e-with-stateful-api: added uuid dev dependency, fixed test
irinakartun Feb 14, 2024
00fd8dd
Merge branch 'development' of github.com:epam/ai-dial-chat into feat/…
irinakartun Feb 14, 2024
1f6a820
feat/update-e2e-with-stateful-api: prettier
irinakartun Feb 14, 2024
35368ab
feat/update-e2e-with-stateful-api: added GPT_4_0125_PREVIEW model
irinakartun Feb 14, 2024
1c77f65
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
54279eb
feat/update-e2e-with-stateful-api: updated paths, removed response wa…
irinakartun Feb 14, 2024
1aa0ecc
feat/update-e2e-with-stateful-api: prettier, fixed authentication fai…
irinakartun Feb 14, 2024
3884695
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 14, 2024
f2c6a2e
feat/update-e2e-with-stateful-api: fixed playwright project's test sets
irinakartun Feb 15, 2024
76b73cd
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 15, 2024
6e1396a
feat/update-e2e-with-stateful-api: fixed flaky tests, added issue ids
irinakartun Feb 15, 2024
880e0a2
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 15, 2024
bffb198
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 15, 2024
74130b0
feat/update-e2e-with-stateful-api: enabled fixed tests, fixed storage…
irinakartun Feb 15, 2024
fcc2156
Merge branch 'development' into feat/update-e2e-with-stateful-api
irinakartun Feb 15, 2024
937a6dc
Merge branch 'development' of github.com:epam/ai-dial-chat into feat/…
irinakartun Feb 15, 2024
7e76878
feat/update-e2e-with-stateful-api: fixed review comment; enabled fixe…
irinakartun Feb 15, 2024
3b56b79
feat/update-e2e-with-stateful-api: fixed test
irinakartun Feb 15, 2024
ee78226
feat/update-e2e-with-stateful-api: fixed test
irinakartun Feb 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion apps/chat-e2e/config/local.playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,13 @@ import { ResultFolder } from '@/src/testData';
import { workspaceRoot } from '@nx/devkit';
import { ReporterDescription } from '@playwright/test';
import dotenv from 'dotenv';
import path from 'path';

/**
* Read environment variables from file.
* https://github.com/motdotla/dotenv
*/
dotenv.config({ path: './.env.development' });
dotenv.config({ path: path.resolve(__dirname, '../../chat/.env.local') });
dotenv.config({ path: './.env.local' });
/**
* Config used for a local run
Expand Down
1 change: 1 addition & 0 deletions apps/chat-e2e/config/playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ export default defineConfig({
},
{
name: 'chromium',
testIgnore: /\/chatApi|listingApi\/.*\.test\.ts/,
use: {
...devices['Desktop Chrome'],
viewport: { width: 1536, height: 864 },
Expand Down
27 changes: 27 additions & 0 deletions apps/chat-e2e/src/core/baseFixtures.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { LocalStorageManager } from '@/src/core/localStorageManager';
import { LoginPage } from '@/src/ui/pages';
import { Auth0Page } from '@/src/ui/pages/auth0Page';
import { test as base } from '@playwright/test';

export const skipReason = 'Execute test on CI env only';

const test = base.extend<{
loginPage: LoginPage;
auth0Page: Auth0Page;
localStorageManager: LocalStorageManager;
}>({
loginPage: async ({ page }, use) => {
const loginPage = new LoginPage(page);
await use(loginPage);
},
auth0Page: async ({ page }, use) => {
const auth0Page = new Auth0Page(page);
await use(auth0Page);
},
localStorageManager: async ({ page }, use) => {
const localStorageManager = new LocalStorageManager(page);
await use(localStorageManager);
},
});

export default test;
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import { DialHomePage, LoginPage } from '../ui/pages';
import { DialHomePage } from '../ui/pages';
import {
Chat,
ChatBar,
ChatHeader,
ChatMessages,
ConversationSettings,
ConversationToCompare,
Conversations,
EntitySelector,
ModelsDialog,
Expand All @@ -13,22 +14,27 @@ import {
RecentEntities,
SendMessage,
} from '../ui/webElements';
import { LocalStorageManager } from './localStorageManager';

import test from '@/src/core/baseFixtures';
import { isApiStorageType } from '@/src/hooks/global-setup';
import { ConversationData } from '@/src/testData';
import {
ChatApiHelper,
FileApiHelper,
IconApiHelper,
} from '@/src/testData/api';
import { ItemApiHelper } from '@/src/testData/api/itemApiHelper';
import { ApiInjector } from '@/src/testData/injector/apiInjector';
import { BrowserStorageInjector } from '@/src/testData/injector/browserStorageInjector';
import { DataInjectorInterface } from '@/src/testData/injector/dataInjectorInterface';
import { PromptData } from '@/src/testData/prompts/promptData';
import { Auth0Page } from '@/src/ui/pages/auth0Page';
import { AccountSettings } from '@/src/ui/webElements/accountSettings';
import { Addons } from '@/src/ui/webElements/addons';
import { AddonsDialog } from '@/src/ui/webElements/addonsDialog';
import { AppContainer } from '@/src/ui/webElements/appContainer';
import { Banner } from '@/src/ui/webElements/banner';
import { ChatInfoTooltip } from '@/src/ui/webElements/chatInfoTooltip';
import { ChatLoader } from '@/src/ui/webElements/chatLoader';
import { Compare } from '@/src/ui/webElements/compare';
import { ConfirmationDialog } from '@/src/ui/webElements/confirmationDialog';
import { DropdownCheckboxMenu } from '@/src/ui/webElements/dropdownCheckboxMenu';
Expand All @@ -50,27 +56,25 @@ import { ShareModal } from '@/src/ui/webElements/shareModal';
import { TemperatureSlider } from '@/src/ui/webElements/temperatureSlider';
import { Tooltip } from '@/src/ui/webElements/tooltip';
import { VariableModalDialog } from '@/src/ui/webElements/variableModalDialog';
import { test as base } from '@playwright/test';
import { allure } from 'allure-playwright';
import path from 'path';
import * as process from 'process';

export const stateFilePath = path.join(
__dirname,
`../../auth/desktopUser${process.env.TEST_PARALLEL_INDEX}.json`,
);
export const stateFilePath = (index: number) =>
path.join(__dirname, `../../auth/desktopUser${index}.json`);

interface ReportAttributes {
setTestIds: (...testId: string[]) => void;
setIssueIds: (...issueIds: string[]) => void;
}

const test = base.extend<
const dialTest = test.extend<
ReportAttributes & {
beforeTestCleanup: string;
dialHomePage: DialHomePage;
loginPage: LoginPage;
auth0Page: Auth0Page;
appContainer: AppContainer;
chatBar: ChatBar;
chatLoader: ChatLoader;
header: Header;
accountSettings: AccountSettings;
accountDropdownMenu: DropdownMenu;
Expand All @@ -93,7 +97,6 @@ const test = base.extend<
addonsDialog: AddonsDialog;
conversationData: ConversationData;
promptData: PromptData;
localStorageManager: LocalStorageManager;
conversationDropdownMenu: DropdownMenu;
folderDropdownMenu: DropdownMenu;
promptDropdownMenu: DropdownMenu;
Expand All @@ -106,6 +109,7 @@ const test = base.extend<
chatInfoTooltip: ChatInfoTooltip;
compare: Compare;
compareConversationSelector: ModelSelector;
compareConversation: ConversationToCompare;
rightConversationSettings: ConversationSettings;
leftConversationSettings: ConversationSettings;
rightChatHeader: ChatHeader;
Expand All @@ -125,6 +129,10 @@ const test = base.extend<
iconApiHelper: IconApiHelper;
chatApiHelper: ChatApiHelper;
fileApiHelper: FileApiHelper;
itemApiHelper: ItemApiHelper;
browserStorageInjector: BrowserStorageInjector;
apiInjector: ApiInjector;
dataInjector: DataInjectorInterface;
}
>({
// eslint-disable-next-line no-empty-pattern
Expand All @@ -141,23 +149,26 @@ const test = base.extend<
const callback = (...issueIds: string[]) => {
for (const issueId of issueIds) {
allure.issue(issueId, `${process.env.ISSUE_URL}/${issueId}`);
test.skip();
dialTest.skip();
}
};
await use(callback);
},
beforeTestCleanup: [
async ({ dataInjector }, use) => {
await dataInjector.deleteAllData();
await use('beforeTestCleanup');
},
{ scope: 'test', auto: true },
],
// eslint-disable-next-line no-empty-pattern
storageState: async ({}, use) => {
await use(stateFilePath(+process.env.TEST_PARALLEL_INDEX!));
},
dialHomePage: async ({ page }, use) => {
const dialHomePage = new DialHomePage(page);
await use(dialHomePage);
},
loginPage: async ({ page }, use) => {
const loginPage = new LoginPage(page);
await use(loginPage);
},
auth0Page: async ({ page }, use) => {
const auth0Page = new Auth0Page(page);
await use(auth0Page);
},
appContainer: async ({ dialHomePage }, use) => {
const appContainer = dialHomePage.getAppContainer();
await use(appContainer);
Expand All @@ -166,6 +177,10 @@ const test = base.extend<
const chatBar = appContainer.getChatBar();
await use(chatBar);
},
chatLoader: async ({ appContainer }, use) => {
const chatLoader = appContainer.getChatLoader();
await use(chatLoader);
},
header: async ({ appContainer }, use) => {
const header = appContainer.getHeader();
await use(header);
Expand Down Expand Up @@ -316,10 +331,6 @@ const test = base.extend<
const promptData = new PromptData();
await use(promptData);
},
localStorageManager: async ({ page }, use) => {
const localStorageManager = new LocalStorageManager(page);
await use(localStorageManager);
},
chatInfoTooltip: async ({ page }, use) => {
const chatInfoTooltip = new ChatInfoTooltip(page);
await use(chatInfoTooltip);
Expand All @@ -328,10 +339,13 @@ const test = base.extend<
const compare = chat.getCompare();
await use(compare);
},
compareConversationSelector: async ({ compare }, use) => {
const compareConversationSelector = compare
.getConversationToCompare()
.getConversationSelector();
compareConversation: async ({ compare }, use) => {
const compareConversation = compare.getConversationToCompare();
await use(compareConversation);
},
compareConversationSelector: async ({ compareConversation }, use) => {
const compareConversationSelector =
compareConversation.getConversationSelector();
await use(compareConversationSelector);
},
rightConversationSettings: async ({ compare }, use) => {
Expand Down Expand Up @@ -386,6 +400,26 @@ const test = base.extend<
const fileApiHelper = new FileApiHelper(request);
await use(fileApiHelper);
},
itemApiHelper: async ({ request }, use) => {
const conversationApiHelper = new ItemApiHelper(request);
await use(conversationApiHelper);
},
apiInjector: async ({ itemApiHelper }, use) => {
const apiInjector = new ApiInjector(itemApiHelper);
await use(apiInjector);
},
browserStorageInjector: async ({ localStorageManager }, use) => {
const browserStorageInjector = new BrowserStorageInjector(
localStorageManager,
);
await use(browserStorageInjector);
},
dataInjector: async ({ apiInjector, browserStorageInjector }, use) => {
const dataInjector = isApiStorageType
? apiInjector
: browserStorageInjector;
await use(dataInjector);
},
});

export default test;
export default dialTest;
24 changes: 9 additions & 15 deletions apps/chat-e2e/src/core/localStorageManager.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
import { Conversation } from '@/chat/types/chat';
import { FolderInterface } from '@/chat/types/folder';
import { Prompt } from '@/chat/types/prompt';
import { Settings } from '@/chat/types/settings';
import { TestConversation, TestFolder, TestPrompt } from '@/src/testData';
import { Page } from '@playwright/test';

export class LocalStorageManager {
Expand All @@ -27,58 +25,54 @@ export class LocalStorageManager {
window.localStorage.setItem('selectedConversationIds', selected);
};

setOpenedFoldersIdsKey = () => (folders: string) => {
window.localStorage.setItem('openedFoldersIds', folders);
};

setSettingsKey = () => (settings: string) => {
window.localStorage.setItem('settings', settings);
};

async setConversationHistory(...conversation: Conversation[]) {
async setConversationHistory(...conversation: TestConversation[]) {
await this.page.addInitScript(
this.setConversationHistoryKey(),
JSON.stringify(conversation),
);
}

async updateConversationHistory(...conversation: Conversation[]) {
async updateConversationHistory(...conversation: TestConversation[]) {
await this.page.evaluate(
this.setConversationHistoryKey(),
JSON.stringify(conversation),
);
}

async setSelectedConversation(...conversation: Conversation[]) {
async setSelectedConversation(...conversation: TestConversation[]) {
await this.page.addInitScript(
this.setSelectedConversationKey(),
JSON.stringify(conversation.map((c) => c.id)),
);
}

async updateSelectedConversation(...conversation: Conversation[]) {
async updateSelectedConversation(...conversation: TestConversation[]) {
await this.page.evaluate(
this.setSelectedConversationKey(),
JSON.stringify(conversation.map((c) => c.id)),
);
}

async setFolders(...folders: FolderInterface[]) {
async setFolders(...folders: TestFolder[]) {
await this.page.addInitScript(
this.setFoldersKey(),
JSON.stringify(folders),
);
}

async updateFolders(...folders: FolderInterface[]) {
async updateFolders(...folders: TestFolder[]) {
await this.page.evaluate(this.setFoldersKey(), JSON.stringify(folders));
}

async setPrompts(...prompt: Prompt[]) {
async setPrompts(...prompt: TestPrompt[]) {
await this.page.addInitScript(this.setPromptsKey(), JSON.stringify(prompt));
}

async updatePrompts(...prompt: Prompt[]) {
async updatePrompts(...prompt: TestPrompt[]) {
await this.page.evaluate(this.setPromptsKey(), JSON.stringify(prompt));
}

Expand Down
3 changes: 3 additions & 0 deletions apps/chat-e2e/src/hooks/global-setup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ import { ResultFolder } from '@/src/testData';
import { FileUtil } from '@/src/utils';
import path from 'path';

export const isApiStorageType =
process.env.STORAGE_TYPE === 'api' || process.env.STORAGE_TYPE === undefined;

export const ExecutionResults = {
allureReportPath: path.resolve(
__dirname,
Expand Down
5 changes: 2 additions & 3 deletions apps/chat-e2e/src/testData/api/chatApiHelper.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
import { Conversation } from '@/chat/types/chat';
import { API } from '@/src/testData';
import { API, TestConversation } from '@/src/testData';
import { BaseApiHelper } from '@/src/testData/api/baseApiHelper';

export class ChatApiHelper extends BaseApiHelper {
public async postRequest(conversation: Conversation) {
public async postRequest(conversation: TestConversation) {
const requestData = {
...conversation,
messages: [conversation.messages[0]],
Expand Down
4 changes: 2 additions & 2 deletions apps/chat-e2e/src/testData/api/fileApiHelper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ export class FileApiHelper extends BaseApiHelper {
public async putFile(filename: string) {
const filePath = path.join(Attachment.attachmentPath, filename);
const bufferedFile = fs.readFileSync(filePath);
const url = `${API.uploadedFileHost()}/${BucketUtil.getBucket()}/${filename}`;
const url = `${API.fileHost}/${BucketUtil.getBucket()}/${filename}`;
const response = await this.request.put(url, {
headers: {
Accept: '*/*',
Expand All @@ -29,7 +29,7 @@ export class FileApiHelper extends BaseApiHelper {
}

public async deleteUploadedFile(filename: string) {
const url = `${API.uploadedFileHost()}/${BucketUtil.getBucket()}/${filename}`;
const url = `${API.fileHost}/${BucketUtil.getBucket()}/${filename}`;
await this.request.delete(url);
}

Expand Down
1 change: 1 addition & 0 deletions apps/chat-e2e/src/testData/api/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ export * from './baseApiHelper';
export * from './chatApiHelper';
export * from './fileApiHelper';
export * from './iconApiHelper';
export * from './itemApiHelper';
Loading
Loading